<html>
<head><meta charset="utf-8"><title>Adding parts of rustc to the performance test suite? · t-compiler/performance · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/index.html">t-compiler/performance</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html">Adding parts of rustc to the performance test suite?</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="210808315"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210808315" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210808315">(Sep 21 2020 at 22:29)</a>:</h4>
<p>What would it take for us to get a point-in-time snapshot of <code>rustc_middle</code> or similar and somehow turn it into a component of the performance test suite?</p>



<a name="210808332"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210808332" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210808332">(Sep 21 2020 at 22:29)</a>:</h4>
<p>I feel like the compiler itself is a <em>huge</em> performance test case that covers certain things the existing performance test suite may not cover as well.</p>



<a name="210808423"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210808423" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210808423">(Sep 21 2020 at 22:30)</a>:</h4>
<p>I'd imagine that part of the issue would be that nightly features change over time and a newer rustc may not always be able to compile an old rustc.</p>



<a name="210808438"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210808438" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210808438">(Sep 21 2020 at 22:30)</a>:</h4>
<p>Also, it might be a challenge to sufficiently isolate a self-contained test case.</p>



<a name="210808448"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210808448" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210808448">(Sep 21 2020 at 22:30)</a>:</h4>
<p>But I think there'd be value in doing so, if we could.</p>



<a name="210809599"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210809599" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210809599">(Sep 21 2020 at 22:45)</a>:</h4>
<p>we used to have this, if you remember irsy, and I'm a bit sad we don't anymore</p>



<a name="210809678"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210809678" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210809678">(Sep 21 2020 at 22:46)</a>:</h4>
<p>I'm not aware of "irsy".</p>



<a name="210809696"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210809696" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210809696">(Sep 21 2020 at 22:46)</a>:</h4>
<p>so <a href="http://perf.rust-lang.org">perf.rust-lang.org</a> is not the first Rust performance collection project</p>



<a name="210809812"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210809812" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210809812">(Sep 21 2020 at 22:48)</a>:</h4>
<p>when I started with Rust, we had irsy ("Is Rust Slim Yet") which gave a combination of time and memory profile (total RSS across the execution of each run, something which <a href="http://perf.rust-lang.org">perf.rust-lang.org</a> doesn't have yet)</p>



<a name="210809872"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210809872" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210809872">(Sep 21 2020 at 22:48)</a>:</h4>
<p>I can try to give some links but it may be broken given how old it is (and ofc it stopped collecting probably before 1.0)</p>



<a name="210809907"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210809907" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210809907">(Sep 21 2020 at 22:49)</a>:</h4>
<p>Oooh, memory usage would be an awesome thing to include in perf.</p>



<a name="210809922"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210809922" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210809922">(Sep 21 2020 at 22:49)</a>:</h4>
<p>we do have max RSS but not anything like irsy did</p>



<a name="210809935"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210809935" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210809935">(Sep 21 2020 at 22:49)</a>:</h4>
<p>OTOH we have many more benchmarks whereas irsy only used rustc itself</p>



<a name="210809966"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210809966" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210809966">(Sep 21 2020 at 22:50)</a>:</h4>
<p>Where does RSS appear?</p>



<a name="210810008"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810008" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810008">(Sep 21 2020 at 22:50)</a>:</h4>
<p>anyway regarding using rustc as a benchmark, on top of the nightly features you mention, there's also the fact that it would add even more to the perf collection time</p>



<a name="210810024"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810024" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810024">(Sep 21 2020 at 22:50)</a>:</h4>
<p><span class="user-mention" data-user-id="239881">@Josh Triplett</span> on <a href="http://perf.rust-lang.org">perf.rust-lang.org</a> you always choose what to view. the default is <code>instructions:u</code> but there's a handful of things measured</p>



<a name="210810044"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810044" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810044">(Sep 21 2020 at 22:50)</a>:</h4>
<p>there's usually a downdrop somewhere</p>



<a name="210810067"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810067" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810067">(Sep 21 2020 at 22:51)</a>:</h4>
<p>e.g. <a href="https://perf.rust-lang.org/?stat=max-rss">https://perf.rust-lang.org/?stat=max-rss</a></p>



<a name="210810078"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810078" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810078">(Sep 21 2020 at 22:51)</a>:</h4>
<p>I'm not seeing such a dropdown.</p>



<a name="210810090"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810090" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810090">(Sep 21 2020 at 22:51)</a>:</h4>
<p>I've seen URLs like that before, but not how to get to them from a normal perf report page.</p>



<a name="210810163"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810163" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810163">(Sep 21 2020 at 22:52)</a>:</h4>
<p>(other than hand-editing the URL)</p>



<a name="210810445"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810445" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810445">(Sep 21 2020 at 22:56)</a>:</h4>
<p><span class="user-mention" data-user-id="239881">@Josh Triplett</span> isn't there a drop-down at the top of the page that says <code>instructions:u</code>?</p>



<a name="210810450"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810450" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810450">(Sep 21 2020 at 22:56)</a>:</h4>
<p>if not can you screenshot the page?</p>



<a name="210810512"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810512" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810512">(Sep 21 2020 at 22:57)</a>:</h4>
<p>Oh, I just saw the dropdown at the <em>bottom</em> of the page, next to the option to start a new comparison.</p>



<a name="210810529"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810529" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810529">(Sep 21 2020 at 22:57)</a>:</h4>
<p>oh if you're looking at a perf result then yeah</p>



<a name="210810539"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810539" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810539">(Sep 21 2020 at 22:57)</a>:</h4>
<p>it should be at the top on <a href="http://perf.rust-lang.org">perf.rust-lang.org</a> itself though</p>



<a name="210810595"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810595" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810595">(Sep 21 2020 at 22:58)</a>:</h4>
<p>The most common way I end up on perf is via a link from <code>@rust-timer</code>. :)</p>



<a name="210810615"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810615" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810615">(Sep 21 2020 at 22:58)</a>:</h4>
<p>it's not really "new comparison", other than being perhaps presented wrong. IMO it should also be at the top</p>



<a name="210810675"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810675" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810675">(Sep 21 2020 at 22:59)</a>:</h4>
<p>it sort of serves dual function I guess though I never use it to change the commit hashes, since it's much easier to do that in the URL</p>



<a name="210810861"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810861" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810861">(Sep 21 2020 at 23:01)</a>:</h4>
<p>anyway it's a bit hard to demo how irsy worked but you can kind of see here <a href="/user_uploads/4715/sImX2VDj_pCKbA1ITRLCEZ0l/image.png">image.png</a></p>
<div class="message_inline_image"><a href="/user_uploads/4715/sImX2VDj_pCKbA1ITRLCEZ0l/image.png" title="image.png"><img src="/user_uploads/4715/sImX2VDj_pCKbA1ITRLCEZ0l/image.png"></a></div>



<a name="210810879"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810879" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810879">(Sep 21 2020 at 23:01)</a>:</h4>
<p>(I zoomed it to 200% to take that screenshot)</p>



<a name="210810992"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210810992" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210810992">(Sep 21 2020 at 23:02)</a>:</h4>
<p><span class="user-mention silent" data-user-id="119009">eddyb</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F/near/210810615">said</a>:</p>
<blockquote>
<p>it's not really "new comparison", other than being perhaps presented wrong. IMO it should also be at the top</p>
</blockquote>
<p>I agree that it should be at the top.</p>



<a name="210811000"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210811000" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210811000">(Sep 21 2020 at 23:02)</a>:</h4>
<p>That would have helped.</p>



<a name="210811051"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210811051" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210811051">(Sep 21 2020 at 23:03)</a>:</h4>
<p>also one annoying thing on large screens is stuff that's right-aligned (when the rest of the page is left-aligned) though I'm not sure now what was</p>



<a name="210811873"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210811873" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> LingM <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210811873">(Sep 21 2020 at 23:14)</a>:</h4>
<p>the dropdown is on the left while the submit "button" to actually switch the metric is all the way on the right</p>



<a name="210811893"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210811893" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210811893">(Sep 21 2020 at 23:14)</a>:</h4>
<p>ah yeah that must've been it, thanks!</p>



<a name="210812752"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210812752" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> LingM <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210812752">(Sep 21 2020 at 23:25)</a>:</h4>
<p>zulip, <a href="/user_uploads/4715/8UTe0JAbmiwN9hAimtfbOW_1/Screenshot-from-2020-09-22-01-23-31.png">you're drunk</a></p>
<div class="message_inline_image"><a href="/user_uploads/4715/8UTe0JAbmiwN9hAimtfbOW_1/Screenshot-from-2020-09-22-01-23-31.png" title="you're drunk"><img src="/user_uploads/4715/8UTe0JAbmiwN9hAimtfbOW_1/Screenshot-from-2020-09-22-01-23-31.png"></a></div>



<a name="210812827"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210812827" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210812827">(Sep 21 2020 at 23:26)</a>:</h4>
<p>I would <em>love</em> to have this</p>



<a name="210812872"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210812872" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210812872">(Sep 21 2020 at 23:27)</a>:</h4>
<p>I think artificial benchmarks are a great way to make you game benchmarks, gaming rustc itself sounds like a much better way to spend time ;)</p>



<a name="210812946"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210812946" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210812946">(Sep 21 2020 at 23:28)</a>:</h4>
<p>The other good benchmark would be incremental for <code>rustc_driver</code> and <code>interface</code> since it's recompiled so much</p>



<a name="210813038"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210813038" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210813038">(Sep 21 2020 at 23:29)</a>:</h4>
<p>Actually, I know perf has a limited time budget so there's only so much it can do per run - could we collect these stats on CI?</p>



<a name="210813056"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210813056" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210813056">(Sep 21 2020 at 23:29)</a>:</h4>
<p>Then we get the data for ~free and also much more frequently</p>



<a name="210813110"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210813110" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210813110">(Sep 21 2020 at 23:30)</a>:</h4>
<p>Only full builds but it would still be enormously useful</p>



<a name="210813171"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210813171" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210813171">(Sep 21 2020 at 23:31)</a>:</h4>
<p>Not sure how much overhead perf stats have, if it's a lot we could only run on PR builds so it doesn't slow down bors</p>



<a name="210813261"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210813261" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210813261">(Sep 21 2020 at 23:32)</a>:</h4>
<p>That also neatly sidesteps the "only builds on beta" issue by only building with beta ;)</p>



<a name="210813447"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210813447" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210813447">(Sep 21 2020 at 23:35)</a>:</h4>
<p>I am currently looking at adding <em>current</em> rustc to perf</p>



<a name="210813469"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210813469" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210813469">(Sep 21 2020 at 23:35)</a>:</h4>
<p>I think a snapshot will be quite painful to backport nightly breakage fixes to</p>



<a name="210813529"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210813529" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210813529">(Sep 21 2020 at 23:36)</a>:</h4>
<p>I think we should have budget to do this</p>



<a name="210813660"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210813660" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210813660">(Sep 21 2020 at 23:38)</a>:</h4>
<p>Current is even better, then we can see how changes affected compile time for bootstrapping, not just whether rustc is slower to run</p>



<a name="210813780"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210813780" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210813780">(Sep 21 2020 at 23:40)</a>:</h4>
<p>I read an article a while back about a codebase (maybe D?) that would only accept new optimizations if they made compiling the compiler itself faster even with the added code <span aria-label="laughing" class="emoji emoji-1f606" role="img" title="laughing">:laughing:</span></p>



<a name="210815795"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210815795" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210815795">(Sep 22 2020 at 00:10)</a>:</h4>
<p>I don't think we'll go that far :)</p>
<p>But I think incremental perf is not going to be a first goal -- modifying the codebase and recompiling is hard(er).</p>



<a name="210815845"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Adding%20parts%20of%20rustc%20to%20the%20performance%20test%20suite%3F/near/210815845" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Adding.20parts.20of.20rustc.20to.20the.20performance.20test.20suite.3F.html#210815845">(Sep 22 2020 at 00:10)</a>:</h4>
<p>sure, just tracking full builds at all would be a major improvement :)</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>